Introducing Linggle: From Concordance to Linguistic Search Engine

We introduce a Web-scale linguistics search engine, Linggle, that retrieves lexical bundles in response to a given query. Unlike a typical concordance, Linggle accepts queries with keywords, wildcard, wild part of speech (PoS), synonymous words, and additional regular expression (RE) operators, and returns bundles with frequency counts. In our approach, we argument Google Web 1T corpus with inverted file indexing, PoS information from BNC, and semantic indexing based on Latent Dirichlet Allocation. The method involves parsing the query to transforming the query to several keyword retrieval commands, retrieving word chunks with counts, filtering the chunks again the query as a RE, and finally displaying the results according the count, similarity, and topic. Clusters of synonymous or conceptually related words are also provided. In addition, Linggle provide example sentences from The New York Times on demand. The current implementation of Linggle is the most comprehensive functionally, and is in principle language and dataset independent. We plan to extend Linggle to provide a fast and convenient access to a wealth of linguistic information embodied in Web scale datasets including Google Web 1T and Google Books Ngram for many major languages in the World. For non-native speakers, doubts concerning the usage of a preposition, the mandatory presence of a determiner, the correctness of the association of a verb with an object or the need for synonyms of a term in a given context are problems that arise frequently when writing in English. Printed collocation dictionaries and reference tools based on compiled corpora offer limited coverage of word usage while knowledge of collocations is vital for the competent use of a language. We propose to address these limitations with a comprehensive system that truly aims at letting learners “know a word by the company it keeps”. Linggle (linggle.com) is a broad coverage language reference tool for English as Second Language learners (ESL). The system is designed to access words in context under various forms. First, we build inverted file index for the Google Web 1T Ngram to support queries with RE-like patterns including PoS and synonym matches. For example, for the query “$V $D +important role”, Linggle retrieve 4-gram chunks that start with a PACLIC-27